On the Effects of Adversarial Perturbations on Distribution Robustness
Yipei Wang, Zhaoying Pan ∗, Xiaoqian Wang · 2026
Read On the Effects of Adversarial Perturbations on Distribution Robustness by Yipei Wang, Zhaoying Pan ∗, Xiaoqian Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16464.txt.
Introduction
Yipei Wang, Zhaoying Pan and Xiaoqian Wang test how foe tweaks bend spread strength. Their 2026 preprint deals with two kinds of model strength. One kind stands firm to small foe tweaks of inputs. The next stands firm when test data shifts in kind. Past work found a trade twixt the two kinds of strength.
The team asks how foe tweaks in train shape that trade. Tweak trained models may gain one strength and lose the next. Clear trade maps help teams pick train terms by need. High stakes tools need both strengths in fair share. The work adds terms to the trade tale.
Catalog record
| CREATOR | Yipei Wang, Zhaoying Pan ∗, Xiaoqian Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16464
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On the Effects of Adversarial Perturbations on Distribution Robustness Yipei Wang, Zhaoying Pan ∗, Xiaoqian Wang Abstract Adversarial robustness refers to a model’s ability to resist perturbation of inputs, while distribution robustness evaluates the performance of the model under data shifts. Although both aim to ensure reliable performance, prior work has revealed a tradeoff in distribution and adversarial robustness. Specifically, adversarial training might increase reliance on spurious features, which can harm distribution robustness, especially the performance on some underrepresented subgroups. We present a theoretical analysis of adversarial and distribution robustness that provides a tractable surrogate for per-step adversarial training by studying models trained on perturbed data. In addition to the tradeoff, our work further identified a nuanced phenomenon that ℓ ∞ \ell_{\infty} perturbations on data with moderate bias can yield an increase in distribution robustness. Moreover, the gain in distribution robustness remains on highly skewed data when simplicity bias induces reliance on the core feature, characterized as greater feature separability. Our theoretical analysis extends the understanding of the tradeoff by highlighting the interplay of the tradeoff and the feature separability. Despite the tradeoff that persists in many cases, overlooking the role of feature separability may lead to misleading conclusions about robustness. 1 Introduction Despite machine learning having achieved great success, its application in the real world can face many challenges, such as input noise ( 9 ) or data shifts ( 26 ) . The well-established datasets are always curated with clean data and labels. At the same time, the real-world input can be accompanied by various image corruptions, such as noise, blur, or changes in brightness ( 10 ) . What’s worse, the models can be sensitive to small adversarial perturbations ( 30 ) that are generated according to the trained models on intention, which results in a minimal change in the image appearance but significantly misleads the model prediction. Therefore, such model vulnerabilities call for effective defense strategies 8 ; 22 to improve adversarial robustness, commonly achieved by incorporating adversarial perturbations in the training data. Figure 1: Examples of Distribution Robustness and Adversarial Perturbations. Groups are denoted by their two binary attributes. The left subplots show distribution robustness when training data contains spurious correlation ( \zeta=0.95 ; 95% majority) and test data is balanced, leading to an accuracy drop under shift. The right subplots show training data perturbed with \ell_{\infty} at varying feature separabilities and a large attack budget of 1 to highlight the effects. When the spurious feature is easier to classify, as m_{1}<m_{2} , the classifier increasingly relies on the spurious feature (feature 2); conversely, m_{1}>m_{2} increases the reliance on the non-spurious feature. In addition to adversarial robustness, robustness to data shift ( 26 ; 14 ) is another major concern in real-world applications. It is common to assume that the training and test data are independent and identically distributed (i.i.d.), but it might fail in real-world applications. This disparity between the training and test data, i.e., distribution shifts ( 14 ) , could likely yield a performance drop on the test data or subgroups that are underrepresented in training data. Ideally, the model is expected to learn the core feature to achieve distribution robustness, by making correct predictions even in the presence of distribution shifts between training and testing data ( 1 ) . However, spurious correlation ( 28 ; 6 ) , where the core feature correlates with other features, poses significant challenges in distribution robustness. For data with spurious correlation, there exist context-dependent cues ( 2 ; 6 ; 33 ) that show high correlation with the groundtruth labels in the training data but are not necessarily required for the groundtruth, and do not even hold in some subgroups. A canonical example, Waterbirds ( 28 ) , is a dataset for bird species classification of water or land birds, where the training data shows a high correlation between the bird species and backgrounds. However, the background does not guarantee that the bird species and models perform poorly on those groups without this correlation. While both adversarial and distribution robustness matter to ensure reliable applications, they are often discussed separately. 24 firstly studied the interaction of adversarial and natural distribution robustness, and formulated it as an explicit tradeoff. In particular, adversarial training, despite being considered to improve adversarial robustness effectively 8 ; 22 , actually leads to a decrease in distribution robustness. Specifically, this work showed that this tradeoff happens with \ell_{1},\ell_{2} adversarial training, along with \ell_{\infty} conditioned on a larger scale for the spurious feature. However, the conditions underlying this tradeoff remain unclear, as the loss-based analysis for adversarial training in ( 24 ) does not fully characterize model performance. To address this gap, we study the effects of adversarial perturbations with a two-stage framework that enables a more precise examination of the interaction between adversarial robustness and distribution robustness. Specifically, we first fit a classifier, then generate adversarial perturbations using its gradients and fit a second classifier on the perturbed data. Motivated by the feature-scale conditions studied in ( 24 ) , we aim to analyze these roles in greater depth. Building on 32 , which hypothesized feature separability as a key factor in spurious correlation modeling and analyzed its impact on accuracy, we adopt a similar perspective in our setting. By examining how the accuracy of the perturbed-data classifier shifts from the clean-data classifier, we investigate the effects and patterns of adversarial perturbations. Following ( 24 ) , we evaluate distribution robustness by considering skewed training data with spurious correlations, where performance drop happens when the test distribution shifts, for example, to balanced test data, as shown in the two left subplots of fig. 1 . Our theoretical results reveal that both the feature separability and the spurious correlation severity play a vital role in the effects of adversarial perturbations on distribution robustness. We show that, beyond the conventional tradeoff, adversarial perturbations can improve distribution robustness in specific regimes. In particular, for moderately biased data where spurious features are correlated with labels but do not fully dominate, \ell_{\infty} perturbations mitigate reliance on spurious features across a broad range of feature separability. This improvement arises even in an extreme spurious correlation when the core features are easy to learn. We validate these theoretical observations using synthetic Gaussian data constructed to match our analytical setting. The experiments are first conducted under the proposed two-stage framework and the results closely match our theoretical insights. Moreover, we conduct the experiments with a standard adversarial training where perturbations are applied per training step, and the results confirm that our two-stage framework serves as an effective and tractable surrogate to study the effects of adversarial training via perturbations. • We present a theoretical analysis, under a simplified framework, to characterize how adversarial perturbations influences distribution robustness. • We show that feature separability and the severity of spurious correlation are key factors to determine the tradeoff between adversarial and distribution robustness. Moreover, we identified more nuanced patterns of their interplay beyond the tradeoff. • Our experiments on synthesized data validate our theoretical insights and also show that our simplified framework generalizes the insights of adversarial perturbations to the standard setup of adversarial training. 2 Related Work Spurious correlation