Batch Normalization Amplifies Memorization and Privacy Risks

Ngoc Phu Doan, Chongyan Gu, Ihsen Alouani · 2026

Read Batch Normalization Amplifies Memorization and Privacy Risks by Ngoc Phu Doan, Chongyan Gu, Ihsen Alouani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24420.txt.

Introduction

Doan, Gu, and Alouani show that batch norm lifts memory of odd samples and leak risks. Batch norm is widely used for faster, calmer training, but its leak effects were new. Across datasets and designs, it lifts memory of odd ones, seen through out-of-spread memory, per-sample pull, and member attacks. That extra memory turns straight into leak weakness, with batch-norm models far more open to member attacks. A gear-level look at the backward pass shows it lifts margin growth for odd samples.

Catalog record

CREATORNgoc Phu Doan, Chongyan Gu, Ihsen Alouani
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63076
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24420

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Batch Normalization Amplifies Memorization and Privacy Risks Ngoc Phu Doan, Chongyan Gu, Ihsen Alouani Abstract Batch Normalization (BN) is widely adopted to enable faster convergence and more stable training of deep neural networks. However, its impact on privacy and memorization has remained largely unexplored. In this work, we investigate the effect of BN layers on the memorization of atypical or outlier samples and its implications for privacy leakage. We conduct an extensive empirical study using three complementary approaches: (i) unintended memorization of out-of-distribution training samples, (ii) per-sample influence measured via gradient norms, and (iii) susceptibility to membership inference attacks (MIA). Across multiple datasets and architectures, we consistently observe that BN substantially increases the memorization of outliers compared to models without BN. Critically, this amplified memorization translates directly into privacy vulnerabilities: models with BN exhibit significantly higher susceptibility to MIAs. We complement our empirical findings with a theoretical analysis showing that BN amplifies the per-step influence of outlier samples during training, providing mechanistic insight into this phenomenon. Our results highlight an underappreciated privacy risk associated with BN and provide both practical and theoretical insights into how normalization layers can amplify the influence of rare or sensitive training examples. 1 Introduction Batch normalization (BN) has become a cornerstone in modern deep learning architectures, normalizing activations across intermediate layers to enhance training stability and overall model performance. It has shown consistent improvement in models’ accuracy and training convergence speed, making it a fundamental component of state-of-the-art architectures. Nevertheless, despite its well-established advantages and pervasive use, the deeper implications of BN on the behavior and properties of deep neural networks remain poorly understood. The original motivation for BN, reducing internal covariance shift [ 22 ] , has been disputed [ 33 ] . The hypothesis that normalization layers enhance smoothness is supported through both empirical and theoretical analyses [ 33 , 5 , 28 ] , while the hessian analysis in [ 43 ] disputes this hypothesis and shows that BN does not necessarily make the loss landscape smooth. Beyond optimization dynamics, recent work has explored BN’s implications for model behavior. For example, several works have investigated the impact of BN on the models’ robustness to adversarial perturbations [ 38 , 44 , 25 ] , eventually concluding a negative impact of BN with an increasing vulnerability to adversarial noise. This suggests that BN might have unintended side effects, influencing the way models represent and respond to data. While robustness has attracted substantial attention, the privacy implications of BN remain largely overlooked. In particular, the relationship between BN and memorization has not been systematically studied. Memorization constitutes a primary avenue for privacy leakage in machine learning, underpinning attacks like membership inference (MIA) [ 34 , 9 ] and sensitive information extraction [ 7 ] . Understanding whether BN accelerates memorization is therefore a critical step in assessing its broader security and privacy consequences. Motivation: Why Outlier Memorization Matters? Prior work has established that memorization in deep learning is not uniformly distributed across the training set: outliers, rare samples, and mislabeled data are disproportionately likely to be memorized [ 14 , 6 , 10 ] . From a privacy perspective, this phenomenon is particularly concerning because outliers often contain sensitive information such as individuals with rare medical conditions, minority demographic characteristics, or unique behavioral patterns, precisely the cases where privacy protection is most critical. In this work, we present the first comprehensive study that establishes a link between BN and memorization and privacy leakage, particularly for outliers and rare samples. Specifically, our results demonstrate that Batch Normalization both accelerates memorization of these vulnerable samples and amplifies their leakage under membership inference attacks. This raises a critical concern: if BN further emphasizes the very samples that are most vulnerable to memorization, then its widespread use in modern architectures may inadvertently increase privacy risks. Contributions. We make the following contributions: • On the empirical side, we provide a systematic evaluation of memorization under three complementary lenses: ( i ) controlled label-flip and out-of-distribution experiments that directly test the model’s propensity to memorize corrupted samples, ( ii ) membership inference attacks that serve as an auditing tool to quantify how training examples leave distinguishable traces in the model, and ( iii ) per-sample influence analysis via gradient-based metrics that reveal the sensitivity of models to individual examples. Together, these approaches offer a multifaceted view of memorization in BN versus noBN models. • We found that Batch Normalization consistently leads to faster and stronger memorization of outliers, as well as increased leakage under membership inference attacks. • On the theoretical side, we provide a rationale why BN disproportionately accelerates the memorization of outlier or noisy samples. We prove that BN amplifies per-step margin growth on tail or mislabeled samples by a factor of (\gamma/\sigma)^{2} . Empirically, we found that (\gamma/\sigma)>1 across a wide range of datasets and models. This quadratic amplification is particularly pronounced for channels with low variance relative to their learned scale parameter. These results reveal a fundamental tension between optimization benefits and privacy guarantees in modern deep learning. Our findings suggest practitioners should carefully weigh these tradeoffs when deploying BN in privacy-sensitive applications. 2 Background Batch Normalization (BN). BN has been widely used in Deep Neural Networks (DNNs) since it accelerates the training convergence and reduces the internal covariance shift to stabilize learning dynamics [ 22 ] . BN intuitively normalizes the data representation space of intermediate layers to zero mean distribution to strengthen the discrimination among datapoints, thus making it easier for the succeeding layers to capture patterns. Let \mathcal{B}_{l}=\{z_{1...m}^{l}\} be the mini-batch output of the model at the l^{th} layer. Mini-batch mean and variance are computed during training as \mu_{\mathcal{B}_{l}}=1/m\sum_{i=1}^{m}z_{i} and \sigma_{\mathcal{B}_{l}}^{2}=1/m\sum_{i=1}^{m}(z_{i}-\mu_{\mathcal{B}_{l}})^{2} . BN at l^{th} layer is determined by: y_{i}=BN(\mathcal{B}_{l})=\gamma\frac{z_{i}-\mu_{\mathcal{B}_{l}}}{\sqrt{\sigma_{\mathcal{B}_{l}}^{2}+\epsilon}}+\beta, (1) where \gamma and \beta are the scale and shift learnable parameters, \epsilon>0 is a tiny constant number to avoid division by zero. Memorization. In machine learning, memorization refers to the phenomenon where a model’s prediction on a training example (x_{i},y_{i}) depends directly on that specific example being present in the training set, rather than on generalizable patterns shared across the data distribution [ 14 ] . Prior work shows that outliers, rare, noisy, or mislabeled examples are disproportionately prone to memorization. For instance, [ 7 ] demonstrated that memorized content is often concentrated in atypical or low-probability regions of the input space. Similarly, [ 46 ] and [ 1 ] observed that deep networks first learn simple, high-frequency patterns shared across many samples, and only later resort to memorizing harder or noisy examples. This tail sensitivity underlies both the generalization–memorization tradeoff [ 14 ] and the privacy risks of large-scale models [ 9 ] . Membership Inference Attack. MIA is a privacy attack where an adversary aims to determine whether a particular data point was included in the training set of a model [ 34 , 6 ] . Successful membership inference directly violates data privacy: it reveals that an individual’s sensitive information was used for training, which can have serious consequences in domains such as healthcare (revealing medical history), finance (exposing transaction records), or personal data processing (disclosing user behavior). The attack exploits a fundamental asymmetry: trained models often exhibit lower

Read the complete text