Promoting Shape Bias in CNNs: Frequency-Based and Contrastive Regularization for Corruption Robustness

Robin Narsingh Ranabhat, Longwei Wang, Amit Kumar Patel, KC santosh · 2025

Read Promoting Shape Bias in CNNs: Frequency-Based and Contrastive Regularization for Corruption Robustness by Robin Narsingh Ranabhat, Longwei Wang, Amit Kumar Patel, KC santosh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-11355.txt.

Introduction

Robin Narsingh Ranabhat and 3 co-authors teach image networks to look at shapes instead of textures in this 2025 paper. Convolutional networks classify images well but break under common corruptions, such as noise and weather effects, that people shrug off. A known cause is their habit of leaning on fine texture rather than the outline of the whole object.

Two training penalties push back. One asks the network's features to agree between a photo and a blurred, low-frequency copy of it, starving texture cues. The other uses supervised contrastive learning to organize features around class-consistent, shape-bearing patterns. On the corrupted CIFAR-10 benchmark, both methods improved robustness while leaving clean accuracy intact, suggesting loss design alone can steer networks toward sturdier vision.

Catalog record

CREATORRobin Narsingh Ranabhat, Longwei Wang, Amit Kumar Patel, KC santosh
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-11355

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Promoting Shape Bias in CNNs: Frequency-Based and Contrastive Regularization for Corruption Robustness Robin Narsingh Ranabhat, Longwei Wang, Amit Kumar Patel, KC santosh Abstract Convolutional Neural Networks (CNNs) excel at image classification but remain vulnerable to common corruptions that humans handle with ease. A key reason for this fragility is their reliance on local texture cues rather than global object shapes—a stark contrast to human perception. To address this, we propose two complementary regularization strategies designed to encourage shape-biased representations and enhance robustness. The first introduces an auxiliary loss that enforces feature consistency between original and low-frequency filtered inputs, discouraging dependence on high-frequency textures. The second incorporates supervised contrastive learning to structure the feature space around class-consistent, shape-relevant representations. Evaluated on the CIFAR-10-C benchmark, both methods improve corruption robustness without degrading clean accuracy. Our results suggest that loss-level regularization can effectively steer CNNs toward more shape-aware, resilient representations. 1 Introduction Convolutional Neural Networks have achieved remarkable success on benchmark datasets such as ImageNet and CIFAR-10, demonstrating high accuracy in a wide range of image classification tasks. However, their performance significantly degrades in the presence of common image corruptions, such as noise, blur, compression artifacts, and environmental distortions like fog or frost [ 2 , 1 , 13 , 14 , 15 , 16 , 18 , 22 , 24 , 25 , 26 ] . This brittleness undermines the practical deployment of CNNs in real-world applications, including autonomous driving, medical imaging, and surveillance systems, where input data is rarely clean or curated. Recent work by Geirhos et al. [ 3 ] revealed that standard CNNs trained on datasets like ImageNet predominantly rely on local texture patterns rather than global object shape for making predictions. This observation contrasts sharply with the human visual system, which is known to prioritize shape cues for object recognition. The reliance on texture renders CNNs highly susceptible to corruptions that primarily perturb high-frequency components—textures—while leaving shape information relatively intact. Figure 1: Custom CNN layer which produces a separate reconstruction of original input x based on low-frequency components obtained after Fourier Transform. Activations a and a’ are generated separately from the same Convolution layer. a’ is cached to calculate the Mean-Squared-Loss. Figure 2: Integrating Supervised Contrastive Loss with Regular Cross-Entropy Loss. We hypothesize this enables learning high-level shape features existent within images of same class. Motivated by this perspective, we propose two orthogonal yet complementary regularization strategies that modify the loss function to encourage CNNs to learn representations that may exhibit reduced texture bias and enhanced shape focus. Our approach centers on the principle that appropriate regularization can guide the optimization process toward solutions that generalize better under distribution shift, particularly corruption-induced changes. Our first regularization approach operates in the frequency domain by introducing auxiliary loss terms that enforce consistency between feature activations of original images and their low-frequency filtered counterparts. This regularization mechanism biases the network toward learning features that remain stable across frequency manipulations, potentially favoring shape-relevant information that typically resides in low-frequency components over texture-specific high-frequency details. Our second regularization strategy employs supervised contrastive learning as an additional term in the training objective. Unlike traditional cross-entropy training that focuses solely on output-level classification accuracy, this regularization approach explicitly structures the intermediate feature space by pulling together representations of same-class samples while pushing apart different-class representations. We hypothesize that this representation-level regularization naturally encourages the learning of class-invariant features, which are more likely to be shape-based than texture-based. The key insight is that strategic regularization can indirectly promote beneficial inductive biases without requiring explicit architectural changes or specialized data preprocessing. We evaluate our regularization methods on the CIFAR-10-C benchmark, which applies 19 diverse corruption types to the CIFAR-10 dataset across five severity levels, providing a rigorous testbed for corruption robustness. Our results demonstrate that both regularization strategies improve robustness while maintaining clean accuracy, showing complementary improvements across different corruption types. 2 Related Work 2.1 Robustness and Regularization in CNNs CNNs struggle under distributional shifts caused by common image corruptions. To assess this vulnerability, Hendrycks and Dietterich [ 2 ] introduced the ImageNet-C and CIFAR-10-C benchmarks, applying real-world corruptions like Gaussian noise and motion blur. Their results showed significant performance drops even in state-of-the-art models. In response, numerous regularization strategies have been proposed to enhance model robustness. Classical regularization techniques like dropout [ 3 ] and batch normalization [ 4 ] have been shown to improve generalization, but their specific impact on corruption robustness has been less systematically studied. Data augmentation remains a widely used approach, wherein training data is augmented with synthetic corruptions to increase diversity and reduce overfitting. Notably, methods such as AugMix [ 5 ] have shown promise in this regard. However, these techniques often require careful design and may struggle to generalize across all corruption types. Alternative solutions include neuroscience-inspired architectural enhancements, such as divisive normalization layers [ 7 ] and push-pull inhibition mechanisms [ 8 , 9 , 27 , 28 , 29 , 30 , 32 , 34 , 35 , 6 ] , which aim to emulate biological visual processing. 2.2 Shape vs. Texture Bias Geirhos et al. [ 10 ] demonstrated that conventional CNNs are strongly biased toward texture-based features rather than shape-based cues. This conclusion was drawn using Stylized ImageNet (SIN), a modified version of ImageNet in which textures were randomized via style transfer while preserving object shapes. CNNs trained on SIN showed increased shape bias and improved robustness to corruptions. This observation stands in contrast to human visual perception, which primarily relies on global shape for object recognition. The gap between CNNs and human perception has motivated follow-up studies exploring techniques to enhance shape bias in models. Some approaches include designing networks with greater receptive fields to better capture global structures [ 11 , 36 ] , implementing advanced data augmentation techniques that preserve structural information [ 12 ] and adopting vision transformer architectures, which inherently prioritize global shape due to their patch-based attention mechanisms [ 17 ] . 2.3 Fourier Perspective on Robustness Another lens through which CNN robustness has been analyzed is the frequency domain. Yin et al. [ 19 ] showed that CNNs tend to rely heavily on high-frequency components, which are particularly sensitive to noise and artifacts. This reliance renders models fragile under conditions that perturb fine-grained details. Building on this insight, Wang et al. [ 20 ] proposed training strategies that encourage models to focus on low-frequency components—typically associated with shape—thereby improving generalization and corruption robustness. These findings support the hypothesis that promoting low-frequency, shape-relevant representations can serve as a foundation for robust visual learning. 2.4 Contrastive Learning Contrastive learning has recently gained prominence as a powerful framework for representation learning, particularly in the context of self-supervised learning [ 21 ] . It operates by structuring the latent space such that similar samples (positive pairs) are drawn closer together while dissimilar ones (negative pairs) are pushed apart. This principle has been extended to supervised learning by Khosla et al. [ 23 ] , who demonstrated that incorporating label information into the contrastive objective significantly improves the quality of learned representations compared to standard cross-entropy training. While contrastive learning has predominantly been applied to improve general feature quality and transferability, its potential to

Read the complete text