Window-based Membership Inference Attacks Against Fine-tuned Large Language Models

Yuetian Chen, Yuntao Du, Kaiyuan Zhang, Ashish Kundu, Charles Fleming, Bruno Ribeiro, Ninghui Li · 2026

Read Window-based Membership Inference Attacks Against Fine-tuned Large Language Models by Yuetian Chen, Yuntao Du, Kaiyuan Zhang, Ashish Kundu, Charles Fleming, Bruno Ribeiro, Ninghui Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-02751.txt.

Introduction

Yuetian Chen and six co authors strengthen membership inference attacks, the tests that ask whether a specific text was inside a language model's training data. Most attacks average a model's loss over the whole text, which dilutes the faint, local traces of memorization. The authors argue membership signals concentrate in short stretches, where a model that saw the text behaves measurably differently from one that did not.

Their method slides windows of many sizes across the text. Each window votes on membership by comparing losses between the target model and a reference model, and votes are combined across scales from single tokens to whole phrases. Across eleven data sets the window based comparison beats established baselines clearly, with higher detection scores and several times better performance at low false positive rates. The result underlines how much fine tuned models leak about their training data.

Catalog record

CREATORYuetian Chen, Yuntao Du, Kaiyuan Zhang, Ashish Kundu, Charles Fleming, Bruno Ribeiro, Ninghui Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE108 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-02751

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Window-based Membership Inference Attacks Against Fine-tuned Large Language Models Yuetian Chen, Yuntao Du, Kaiyuan Zhang, Ashish Kundu, Charles Fleming, Bruno Ribeiro, Ninghui Li Abstract Most membership inference attacks (MIAs) against Large Language Models (LLMs) rely on global signals, like average loss, to identify training data. This approach, however, dilutes the subtle, localized signals of memorization, reducing attack effectiveness. We challenge this global-averaging paradigm, positing that membership signals are more pronounced within localized contexts. We introduce WBC (Window-Based Comparison), which exploits this insight through a sliding window approach with sign-based aggregation . Our method slides windows of varying sizes across text sequences, with each window casting a binary vote on membership based on loss comparisons between target and reference models. By ensembling votes across geometrically spaced window sizes, we capture memorization patterns from token-level artifacts to phrase-level structures. Extensive experiments across eleven datasets demonstrate that WBC substantially outperforms established baselines, achieving higher AUC scores and 2–3 × \times improvements in detection rates at low false positive thresholds. Our findings reveal that aggregating localized evidence is fundamentally more effective than global averaging, exposing critical privacy vulnerabilities in fine-tuned LLMs. 1 Introduction Large Language Models (LLMs) have achieved transformative success across numerous applications 94 ; 80 ; 17 . However, the training of these models often involves extensive datasets that can contain private or sensitive information. This practice introduces significant privacy risks, including the memorization and potential leakage of training data 9 ; 45 ; 42 ; 57 . Membership Inference Attacks (MIAs)—which determine whether specific samples were included in training data—are the primary method for quantifying these risks 92 , with successful attacks directly demonstrating information leakage 67 ; 29 ; 108 ; 79 . Figure 1: Performance comparison of membership inference attacks on Pythia-2.8B. ROC curves showing true positive rate vs. false positive rate (both on log scale) for various MIA methods evaluated on Web Samples V2 split. The diagonal dashed line represents random guessing performance. Numbers in legends indicate AUC scores. Our proposed WBC Attack significantly outperforms existing baselines across all false positive rate regimes, demonstrating superior membership inference capability. Figure 2: Overview of the Window-Based Comparison ( WBC ) Attack. Unlike baseline methods (a) that rely on comparing a single, noisy global average of per-token losses, our approach (b) introduces a local aggregation step. We slide a window across the loss sequences from the target ( \mathcal{M}^{\text{T}} ) and reference ( \mathcal{M}^{\text{R}} ) models, making a binary comparison for each window. The final membership score is the sum of this local evidence, a process that filters noise and provides a more sensitive measure, leading to better separation between member and non-member distributions. These privacy vulnerabilities are particularly pronounced during the fine-tuning stage, where models are adapted to specialized, often proprietary, datasets 68 ; 67 ; 11 . Most established MIAs in this setting are reference-based, operating by comparing a global statistic—typically the average per-token loss—between a fine-tuned target model and a pre-trained reference model 100 ; 106 ; 29 ; 3 . In a reference-based MIA, for each instance, one computes token-level losses (negative log predicted probability) from the reference model minus those from the target model. To understand how to best utilize such token loss difference sequences for membership inference, we analyzed the distributions of loss differences for 10 million tokens. Our analysis reveals that membership signals manifest as sparse, extremal events rather than uniform distributional shifts. Furthermore, these membership signals are intermixed with sparser and more extreme events caused by domain-specific tokens, which have high loss reduction and occur with similar frequencies in non-members as well as members. These events create long-tailed noise with potentially infinite variance; global averaging becomes unreliable—a single outlier can dominate the entire statistic, obscuring genuine membership signals concentrated in localized regions. This raises a critical question: Can robust detection methods overcome the fundamental challenge of extracting sparse membership signals from long-tailed noise dominated by domain adaptation effects? Building on the insights gained from the empirical analysis, we construct a mathematical model for the token loss reduction sequences using the point process theory from extreme value statistics 53 ; 26 . Theoretical analysis using the model shows that the most reliable membership signals are concentrated in localized token sequences. While individual token losses are too volatile to be trusted, aggregating them over short windows (e.g., 3-10 consecutive tokens) can filter noise without destroying the underlying signal. Taking advantage of these theoretical insights, we introduce the Window-Based Comparison ( WBC ) attack, illustrated in Figure 2 . Our method employs a sliding window approach that performs hundreds of local comparisons, combined with sign-based aggregation that counts the fraction of windows favoring membership. Our contributions are as follows: 1. We are the first to use empirical analysis of distributions of token-level loss signals to understand how to construct more effective MIAs. Our analysis results in several intriguing findings. For example, counter-intuitively, the strongest membership signals occur on tokens where the fine-tuned model has a higher loss than the reference model. See Section 4.1.1 for discussion of these findings. We conjecture that a similar analysis could result in useful insights for MIA in pre-trained LLMs and possibly other paradigms, including vision-language 81 and diffusion language models 74 . 2. We formalize these observations with a mixture of point process models, which explains why global averaging is suboptimal and provides theoretical grounding for localized detection. We thus propose the WBC attack that replaces global averaging with sliding window analysis. Our method captures localized memorization patterns while maintaining robustness to long-tailed noise. A geometric ensemble strategy aggregates evidence across multiple window sizes, eliminating parameter tuning. 3. Through extensive experiments on eleven diverse datasets using various models, we show that WBC significantly outperforms thirteen baseline attacks. Averaged across all datasets, WBC achieves an AUC of 0.839 compared to the strongest baseline’s 0.754, and improves the True Positive Rate at 1% False Positive Rate by 2.8\times (from 5.2% to 14.6%). Figure 1 shows this superiority on a representative example, the Web Samples V2 dataset. Our findings prove that the aggregation of local signals is a more potent attack vector than previously established global methods. This work not only introduces a more effective MIA but also underscores the need for defenses that can account for these localized memorization patterns. 2 Related Works LLM. Large Language Models (LLMs) build on transformer architectures 97 , employing self-supervised pre-training objectives such as masked language modeling 18 , causal language modeling 83 , or span corruption 84 over vast, diverse corpora to capture linguistic patterns and world knowledge. Guided by empirical scaling laws that correlate model size, dataset volume, and compute with performance 49 ; 39 ; 37 , LLMs have scaled to trillions of parameters, exemplified by foundational models like GPT-3 7 , PaLM 13 , BLOOM 103 , OPT 112 , LLaMA 95 , Mistral 47 , and Falcon 77 . Recent advancements incorporate architectural innovations, including mixture-of-experts (MoE) for efficient scaling 90 ; 27 ; 46 , FlashAttention for optimized memory and speed 15 , and hybrid designs blending dense and sparse activations 55 . State-of-the-art models such as LLaMA-3 25 , Qwen3 105 , DeepSeek-V3 17 , and Gemma-3 93 push

Read the complete text