Data Augmentation Through Random Style Replacement
1 st Qikai Yang * 2 nd Cheng Ji University of Illinois Urbana-Champaign , Urbana, USA University of Illinois Urbana-Champaign , Urbana, USA * Corresponding Author: [email protected] 3 th Huaiying Luo 4 th Panfeng Li Cornell University , Ithaca, USA University of Michigan , Ann Arbor, USA 5 th Zhi · 2025
Read Data Augmentation Through Random Style Replacement by 1 st Qikai Yang * 2 nd Cheng Ji University of Illinois Urbana-Champaign , Urbana, USA University of Illinois Urbana-Champaign , Urbana, USA * Corresponding Author: [email protected] 3 th Huaiying Luo 4 th Panfeng Li Cornell University , Ithaca, USA University of Michigan , Ann Arbor, USA 5 th Zhi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10563.txt.
Introduction
Qikai Yang and co-authors wrote a paper from 2025 on a new way to multiply training images by mixing style transfer with patch swapping. Data augmentation makes models sturdier by showing them varied versions of the same image. Style methods repaint an image in another's look, while erasing methods blank out patches. Their method does both in sequence for a stronger effect.
First a random style transfer repaints the training image. Then random subregions of the original are swapped for the matching patches from the repainted version. The result keeps the scene's content while scrambling its local look. The scheme accepts a wide range of style transfer engines and slots into existing augmentation pipelines. In comparison tests, models trained with the mixed method scored higher than those fed by style methods alone. They also settled into their final skill faster during training.
Catalog record
| CREATOR | 1 st Qikai Yang * 2 nd Cheng Ji University of Illinois Urbana-Champaign , Urbana, USA University of Illinois Urbana-Champaign , Urbana, USA * Corresponding Author: [email protected] 3 th Huaiying Luo 4 th Panfeng Li Cornell University , Ithaca, USA University of Michigan , Ann Arbor, USA 5 th Zhi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-10563
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Data Augmentation Through Random Style Replacement 1 st Qikai Yang * 2 nd Cheng Ji University of Illinois Urbana-Champaign , Urbana, USA University of Illinois Urbana-Champaign , Urbana, USA * Corresponding Author: [email protected] 3 th Huaiying Luo 4 th Panfeng Li Cornell University , Ithaca, USA University of Michigan , Ann Arbor, USA 5 th Zhicheng Ding Columbia University , New York, USA Abstract In this paper, we introduce a novel data augmentation technique that combines the advantages of style augmentation and random erasing by selectively replacing image subregions with style-transferred patches. Our approach first applies a random style transfer to training images, then randomly substitutes selected areas of these images with patches derived from the style-transferred versions. This method is able to seamlessly accommodate a wide range of existing style transfer algorithms and can be readily integrated into diverse data augmentation pipelines. By incorporating our strategy, the training process becomes more robust and less prone to overfitting. Comparative experiments demonstrate that, relative to previous style augmentation methods, our technique achieves superior performance and faster convergence. I Introduction Recent advancements in deep learning have driven significant progress in a wide range of computer vision tasks, including image classification, object detection, and semantic segmentation [ 1 , 2 ] . Despite these advances, many of these tasks continue to face a fundamental bottleneck: a lack of sufficient labeled data [ 3 , 4 ] . Annotating large-scale datasets is both time-consuming and costly, which can limit the applicability of deep neural networks in specialized or rapidly evolving domains. To mitigate this issue, data augmentation techniques are heavily utilized, artificially expanding and diversifying the training set so that models generalize more effectively. (a) (b) (c) (d) (e) Fig. 1 : Examples of random style transfer: we generate a style-transferred image and use it to patch original image in different ways (a) Input Image. (b) Random-Region-Erased Image. (c) Style Transfer. (d) Random Style Replacement. (e) Random Style Replacement. Data augmentation [ 5 ] has garnered significant attention in supervised learning research across a wide range of domains—including computer vision [ 6 , 7 ] , natural language processing [ 8 ] , graph learning [ 9 , 10 ] , and large language model [ 11 , 12 ] — due to its ability to increase both the volume and diversity of training data, thereby enhancing model generalization and mitigating overfitting. Broadly, data augmentation strategies can be grouped into two categories: generative methods, which utilize models like Variational Autoencoders (VAEs) [ 13 ] , Generative Adversarial Networks (GANs) [ 14 ] , Large Language Models (LLMs), or diffusion-based frameworks [ 15 ] to synthesize new data; and traditional methods, which rely on transformations such as random cropping, flipping, rotations, color jittering, and histogram equalization to modify existing samples. While both approaches aim to expose the model to a wider variety of conditions, thus reducing overfitting, traditional augmentation strategies may not fully capture the complexity of real-world variability. Consequently, several studies have explored more refined methods, including style augmentation and random erasing [ 16 , 17 , 18 ] . Style augmentation employs style transfer [ 19 ] to alter the visual attributes of training images while preserving their semantic content, thereby increasing robustness to differences in texture, color, and contrast. Random erasing, on the other hand, randomly occludes or replaces subregions of an image, making models more resilient to missing or corrupted information. In this paper, we revisit these traditional approaches—particularly focusing on their potential to advance the efficacy of data augmentation in supervised learning. In this paper, we introduce a novel data augmentation method that merges style augmentation with random erasing. Our approach involves applying a random style transfer to an image, followed by replacing specific subregions with patches from the style-transferred version. This technique enhances robustness against style variations and occlusion-like effects. It integrates smoothly with existing style transfer frameworks and fits easily into standard data augmentation pipelines, as illustrated in Figure 1 . The main contributions of our work are as below: 1. We propose a technique that merges style augmentation and random erasing, offering benefits from both texture variation and structured occlusion. 2. We demonstrate through experiments that our approach reduces the risk of overfitting while achieving faster convergence compared to established style augmentation methods. Ease of Integration: Our strategy is parameter-free and can be readily adapted to a broad spectrum of computer vision tasks, making it a highly practical solution for data augmentation. By leveraging this new augmentation method, we observe notable gains in model performance across different tasks, highlighting its potential to address the persistent challenge of limited labeled data in computer vision research. II Dataset We tested our random style replacement method on the STL-10 dataset, which includes 5,000 training images and 8,000 test images, each with a resolution of 96×96 pixels across 10 classes [ 3 ] . We chose STL-10 due to its complex backgrounds and high resolution, which pose a substantial challenge for image classification, making it a robust benchmark. Additionally, the limited size of the training set highlights the effectiveness of our data augmentation technique in enhancing training data. III Methods This sections introduces our random style replacement method in details. We described the overall process of random style replacement and explain how we perform image patch. III-A Random Style Replacement During training, random style replacement is applied with a certain probability p : for each image I in a mini-batch, there’s a probability p that it undergoes style replacement and a probability 1-p that it remains unchanged. If selected, the image will be transformed into a new version with a partial style change. This random style replacement process consists of two steps: generating a complete-style-transferred image and merging it with the original image by certain patching methods. The procedure is shown in Alg. 1 . Style transfer refers to a class of image processing algorithms that alter the visual style of an image while preserving its semantic content. For style transfer to be part of a data augmentation technique, it needs to be a both fast and random algorithm capable of applying a broad range of styles. Therefore, we adopt the approach of Jackson et al., which efficiently generates a completely style-transferred image by incorporating randomness on the fly without requiring heavy computations [ 16 ] . The generated style-transferred image will then be used to patch the original image, creating an augmented image. There are multiple patching methods, and we adopt the two most common ones: patching by a random subregion and patching randomly selecting individual pixels. To avoid bias in data augmentation, we employed random style transferring to ensure diverse and uniform modifications across all image types, enhancing model generalization. Algorithm 1 Random Style Replacement Procedure Input : Input image I ; Augmentation probability p ; Patch mode pMode ; Output : Augmented image I^{\ast} . Initialization: p_{1}\leftarrow Rand (0, 1). 1 if p_{1}\geq p then 2 I^{\ast}\leftarrow I ; 3 return I^{\ast} . 4 else 5 I^{{}^{\prime}}\leftarrow\textbf{randomStyleTransfer}(I) ; 6 I^{\ast}\leftarrow\textbf{randomPatch}(I,I^{{}^{\prime}},pMode) ; 7 return I^{\ast} . 8 end if III-B Random Patch Random patch is to patch a image based on another image. Here, we provided a detailed explanation