GenView++: Unifying Adaptive View Generation and Quality-Driven Supervision for Contrastive Representation Learning
Xiaojie Li, Bei Wang, Jianlong Wu, Yue Yu · 2025
Read GenView++: Unifying Adaptive View Generation and Quality-Driven Supervision for Contrastive Representation Learning by Xiaojie Li, Bei Wang, Jianlong Wu, Yue Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23770.txt.
Introduction
Contrastive learning lives or dies on the quality of the pairs it pulls together. Xiaojie Li and 6 co-authors built GenView++ to improve both how pairs are made and how they are weighted. Handcrafted and generative augmentations can lack diversity or corrupt meaning. Their generator adapts across image-only, text-only, and mixed conditioning to synthesize views that stay semantically coherent.
A quality mechanism then scores each pair on alignment and diversity, reweighting training so strong pairs count more and redundant ones less. Gains appear across vision and vision-language tasks, including a clear lift over CLIP on zero-shot classification across ten data sets.
Catalog record
| CREATOR | Xiaojie Li, Bei Wang, Jianlong Wu, Yue Yu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 96 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23770
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GenView++: Unifying Adaptive View Generation and Quality-Driven Supervision for Contrastive Representation Learning Thanks: Xiaojie Li, Bei Wang, Jianlong Wu, Liqiang Nie, and Min Zhang are with the School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), 518055, China. E-mail: [email protected], [email protected], [email protected], [email protected], [email protected]. Xiaojie Li, Jianlong Wu and Yue Yu are with the Pengcheng Laboratory, Shenzhen, China. E-mail: [email protected]. Xiaojie Li, Bei Wang, Jianlong Wu, Yue Yu Abstract The success of contrastive learning depends on the construction and utilization of high-quality positive pairs. However, current methods face critical limitations on two fronts: on the construction side, both handcrafted and generative augmentations often suffer from limited diversity and risk semantic corruption; on the learning side, the absence of a quality assessment mechanism leads to suboptimal supervision where all pairs are treated equally. To tackle these challenges, we propose GenView++, a unified framework that addresses both fronts by introducing two synergistic innovations. To improve pair construction, GenView++ introduces a multi-source adaptive view generation mechanism to synthesize diverse yet semantically coherent views by dynamically modulating generative parameters across image-conditioned, text-conditioned, and image-text-conditioned strategies. Second, a quality-driven contrastive learning mechanism assesses each pair’s semantic alignment and diversity to dynamically reweight their training contribution, prioritizing high-quality pairs while suppressing redundant or misaligned pairs. Extensive experiments demonstrate the effectiveness of GenView++ across both vision and vision–language tasks. For vision representation learning, it improves MoCov2 by +2.5% on ImageNet linear classification. For vision–language learning, it raises the average zero-shot classification accuracy by +12.31% over CLIP and +5.31% over SLIP across ten datasets, and further improves Flickr30k text retrieval R@5 by +3.2%. The code is available at https://github.com/xiaojieli0903/GenViewPlusPlus . I Introduction Fig. 1 : Motivation of GenView++. (a) Standard contrastive learning methods rely on handcrafted augmentations, which provide limited diversity, risk semantic distortions, and lack pair-level quality control during training. (b) GenView++ addresses these limitations via (i) Multi-Source Adaptive View Generation , which synthesizes diverse and semantically aligned views, and (ii) Quality-Driven Contrastive Learning , which reweights supervision by assessing the quality of both image–image and image–text pairs. Self-supervised learning (SSL) has emerged as a fundamental paradigm for acquiring robust and generalizable representations from large-scale unlabeled data. Among various SSL methods, contrastive learning (CL) [ 1 , 2 ] has shown remarkable success in both vision [ 3 , 4 , 5 ] and vision-language [ 6 , 7 ] representation learning. The core principle of CL is to learn invariant and discriminative representations by maximizing the similarity between different views of the same instance (positive pairs) while pushing them apart from other instances. Its effectiveness crucially depends on constructing and utilizing high-quality positive pairs that navigate a critical trade-off: they must be semantically consistent to ensure true invariance is learned, yet sufficiently diverse to foster robust generalization to unseen variations. However, current methods often employ handcrafted augmentations (e.g., random cropping, color jittering) on the same instance to obtain positive views. These transformations offer limited diversity , as they only modify surface-level visual characteristics and fail to introduce new content or high-level variations (e.g., object viewpoints, textures, or variations within a semantic category), often yielding redundant views (Fig. 1 a-1, a-4). Overly aggressive augmentations are not always precise, potentially causing false positive pairs when essential content is cropped out (a-2, a-3). These issues are further exacerbated in vision–language pretraining: redundant views provide limited benefit for visual–text grounding, while distorted views cause semantic mismatches that disrupt image–caption alignment. The limitations of handcrafted augmentations highlight generative models such as Stable Diffusion [ 8 ] as promising alternatives, as their abundant prior knowledge learned from large-scale datasets enables the synthesis of high-quality, diverse images that enrich view contents. However, integrating pretrained generative models into contrastive learning is non-trivial. Despite their strong generative capability, these models lack controllability over generation contents , making it difficult to balance semantic fidelity with diversity and increasing the risk of producing views that deviate from the conditional inputs, thus creating false positive pairs. Moreover, most existing methods rely on a single conditioning source [ 9 , 10 , 11 ] , conditioning on either the image or the text, but not both. This unimodal approach overlooks the powerful synergistic information in image-text pairs, missing the opportunity to synthesize views that are simultaneously consistent with the visual content and aligned with textual semantics. Compounding all these construction side issues is a more fundamental limitation on the learning side : the lack of pair-level quality assessment . Current pipelines treat every training pair as equally valuable, assigning a uniform importance score regardless of its semantic alignment or diversity (as illustrated by the gray scores in Fig. 1 (a)). This indiscriminate supervision forces the model to learn from a mix of high-quality, redundant, and false positive pairs from both aggressive augmentations and noisy web-crawled data [ 12 , 13 ] , ultimately limiting the quality of the final representations. To tackle these fundamental challenges in both construction and learning sides, we propose GenView , a controllable framework that enhances view quality for vision representation learning using the powerful pretrained generative model, and guides contrastive learning via quality assessment. Specifically, given an image as the source view, we construct its positive view using the synthetic image sampled from a pretrained generative model conditioned on this image. To optimally balance the trade-off between diversity and semantic fidelity, we design an adaptive view generation strategy to generate diverse yet semantically consistent views by adaptively perturbing the image embedding according to foreground saliency. Low-saliency subjects receive weaker perturbations to ensure the correct semantic content of the synthetic image, while high-saliency subjects receive stronger perturbations to create more variations for more diverse content and environments. Furthermore, we develop a quality-driven contrastive learning mechanism to mitigate the effect of potential false positive pairs that could mislead contrastive learning. We score each image-image pair considering both foreground similarity and background diversity. It prioritizes the positive pairs with high foreground similarity to ensure semantic coherence, while also favoring the pairs with low background similarity to promote diverse environments for learning invariant representations. We then recalibrate the contrastive loss function by reweighting each pair with its pair quality, which enhances the contributions of high-quality positive pairs, and simultaneously reduces the influence of low-quality and even false pairs. Building upon our prior work, GenView [ 14 ] , published in ECCV 2024 [ 14 ] , which first introduced the principles of adaptive generation and quality-driven supervision for vision-only representation learning, we propose GenView++ , a substantial extension that generalizes these principles into a unified framework for both vision and vision–language representation learning through two synergistic innovations. First , to tackle the pair construction challenges such as limited diversity, lack of controllability, and single-modality conditioning, GenView++ introduces a multi-source adaptive view generation module. This offline mechanism leverages pretrained diffusion models to synthesize a rich spectrum of diverse yet semantically faithful views. It uniquely integrates three complementary and adaptively controlled conditioning strategies: (1) Image-Conditioned mode (Fig. 1 (b-2), inherited from GenView, enhances visual invariance by introducing saliency-aware variations that preserve the core semantics of the original image. (2) Text-Conditioned mode (Fig. 1 (b-3)) leverages captions to guide synthesis, adaptively tuning the guidance strength based on caption complexity: