C-LEAD: Contrastive Learning for Enhanced Adversarial Defense

Suklav Ghosh, Sonal Kumar, Arijit Sur · 2025

Read C-LEAD: Contrastive Learning for Enhanced Adversarial Defense by Suklav Ghosh, Sonal Kumar, Arijit Sur (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27249.txt.

Introduction

Suklav Ghosh, Sonal Kumar, and Arijit Sur test a fresh shield for image AI in this 2025 paper. Attackers can flip a network's answer with changes too small for eyes to see. Most shields train on attacked images alone. Their method, C-LEAD, adds contrastive learning, a style that pulls like images close and pushes unlike ones apart.

The network trains on both clean and attacked images at once. The setup adjusts the model and the attack tweaks side by side. The aim is features, the traits the network reads, that stay steady under attack. In tests, the shield held up well against many attack kinds. The authors say the contrastive loss draws out stronger, more telling traits. Code is shared on GitHub for others to check.

Catalog record

CREATORSuklav Ghosh, Sonal Kumar, Arijit Sur
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-27249

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

C-LEAD: Contrastive Learning for Enhanced Adversarial Defense Suklav Ghosh, Sonal Kumar, Arijit Sur Abstract Deep neural networks (DNNs) have achieved remarkable success in computer vision tasks such as image classification, segmentation, and object detection. However, they are vulnerable to adversarial attacks, which can cause incorrect predictions with small perturbations in input images. Addressing this issue is crucial for deploying robust deep-learning systems. This paper presents a novel approach that utilizes contrastive learning for adversarial defense, a previously unexplored area. Our method leverages the contrastive loss function to enhance the robustness of classification models by training them with both clean and adversarially perturbed images. By optimizing the model’s parameters alongside the perturbations, our approach enables the network to learn robust representations that are less susceptible to adversarial attacks. Experimental results show significant improvements in the model’s robustness against various types of adversarial perturbations. This suggests that contrastive loss helps extract more informative and resilient features, contributing to the field of adversarial robustness in deep learning. keywords Adversarial training, Contrastive learning, Representation Learning, Computer Vision, Deep Learning. 1 Introduction Deep learning is one of the most widely used tools in computer vision research. It enables us to develop deep neural networks like convolutional neural networks (CNN) and vision transformers to perform various computer vision tasks. Before deploying in practical scenarios, these models undergo crucial training and testing on extensive datasets. However, such models are vulnerable to attacks that manipulate predictions by introducing visually imperceptible perturbations to training images and videos, known as adversarial perturbations Ranga and Nageswara Guptha 2023 ; Sarker 2021 . Hence, it’s essential to take certain measurements before utilizing deep learning models for critical computer vision applications. A general framework for adversarial attack is described in Figure 1 . One major problem with using neural networks in safety-paramount applications, such as autonomous driving, has been their susceptibility to miniature perturbations Szegedy et al. 2014 . To guarantee the trained networks’ resilience towards adversarial attacks Zhang et al. 2019 ; Tramèr et al. 2020 ; Madaan et al. 2020 , random noise Zheng et al. 2016 , and corruption Hendrycks and Dietterich 2019 ; Yin et al. 2019 , a number of articles were put forward. In order to achieve the highest possible loss on the target framework, adversarial learning—which trains the framework using perturbed samples—may be among the most often used methods for achieving adversarial resilience. Adversarial learning has advanced significantly through recent years, beginning with the fast gradient sign method(FGSM), which employs a perturbation along the gradient direction, and moving towards projected gradient descent(PGD), which provides the highest loss throughout iterations, and TRADES, which compromises between adversarial robustness and clean accuracy Goodfellow et al. 2015 ; Madry et al. 2018 ; Zhang et al. 2019 . Despite this, in order to produce adversarial attacks, traditional adversarial learning strategies must have class labels. Figure 1: A general framework for the adversarial attack in a deep learning network. Self-supervised learning Gidaris et al. 2018 ; Noroozi and Favaro 2016 ; Chen et al. 2020 ; He et al. 2020 ; Noroozi and Favaro 2017 has gained prominence in the past few years as a method of learning representations for deep neural networks. It involves training the framework on unlabeled data in a supervised way using self-generated labels out of the data itself Carmon et al. 2019 . These self-supervised learning techniques include, for instance, tackling randomized Jigsaw puzzles Noroozi and Favaro 2016 and predicting the angle of rotation Gidaris et al. 2018 . Instance-level identity retention combined with contrastive learning has been demonstrated to be a highly successful method for acquiring rich representations for classification Chen et al. 2020 ; He et al. 2020 . The general goal of contrastive self-supervised learning architectures, like those found in Chen et al. 2020 ; He et al. 2020 ; Noroozi and Favaro 2017 ; Tian et al. 2020 , is to minimize an instance’s closeness to other examples while maximizing its resemblance to its augmentation. The stated contrastive learning is a widely studied representation learning approach Kumar et al. 2023 . A general contrastive learning framework consists of a negative and positive pair sampling strategy, a deep learning model (feature extractor), and a contrastive loss (an objective function). The contrastive loss function minimizes the distance of positive pairs and maximizes the distance of negative pairs in the feature representation space. In our paper, we extend the idea of pair sampling strategy and the objective function of the contrastive learning approach for adversarial defense training. Our modified sampling strategy creates a positive pair by sampling multiple perturbed versions of an image and a negative pair by sampling multiple perturbed versions of different images. For clarification, all images in a positive pair are perturbed versions of the same image, and each image in a negative pair is a perturbed version of different images from the dataset. The intuition is to bring the anchor image and its different perturbed versions close in a feature representation space with a contrastive learning approach. The perturbed versions of images are generated with existing adversarial attack mechanisms like FGSM, PGD, and CW. The deep learning model, pre-trained with our method, can produce a similar representation for an image and its perturb versions. Later, we utilize the robust pre-trained model to perform downstream tasks like image classification. It also acts as a filter for downstream tasks to prevent adversarial attacks. The experimental results show that the proposed contrastive adversarial training makes the feature extractor or the CNN backbone robust enough to handle perturbed images by producing a visual feature representation similar to the anchor images. The major contributions of our paper are: 1. Introduced a novel framework based on contrastive learning to enhance deep learning model robustness, thoroughly investigating various attack techniques and developing a resilient model capable of withstanding both known and unknown gradient-based attacks during training and testing stages. 2. Achieved significant improvements in backbone model performance through experiments: FGSM attack resistance increased by 40%, PGD attack resistance enhanced by 53%, and CW attack resistance improved by 41%. The rest of the paper is organized as follows. Section 2 reviews the Literature, and Section 3 presents the proposed methodology. Section 4 elucidates the results & analysis section, which is followed by the conclusion section. 2 Related Work 2.1 Contrastive Learning In recent research on the learning of metrics Chopra et al. 2005 ; Weinberger and Saul 2009 ; Schroff et al. 2015 , contrastive learning has been applied extensively. In recent years, it has been utilized for self-supervised learning (SSL) van den Oord et al. 2018 ; Noroozi and Favaro 2017 ; Ye et al. 2019 ; Tian et al. 2020 ; He et al. 2019 ; Chen et al. 2020 ; Misra and van der Maaten 2020 ; Sermanet et al. 2017 ; Hyärinen and Morioka 2016 , in which it is employed to learn an encoder during the pretext training phase. The goal of contrastive learning approaches in the self-supervised learning environment, in the absence of labels, is to learn a uniform representation for every image in the training set. In order to accomplish that, a contrastive loss assessed upon pairs of feature vectors taken from

Read the complete text