Membership Inference on LLMs in the Wild
Jiatong Yi, Yanyang LiThe Chinese University of Hong Kong {jtyi2, yyli21}@cse.cuhk.edu.hk · 2026
Read Membership Inference on LLMs in the Wild by Jiatong Yi, Yanyang LiThe Chinese University of Hong Kong {jtyi2, yyli21}@cse.cuhk.edu.hk (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-11314.txt.
Introduction
Jiatong Yi and Yanyang Li test membership attacks on language models in the wild. Their 2026 preprint deals with large language models as found in use. A membership attack asks if one text trained the model. That test audits what opaque training sets truly held.
Most past tests lean on things users cannot see. Logits and other internals are hid by real services. In a strict black-box case, only the output text shows. Tests also tend to fail when the domain shifts. The team's work targets that real-world black-box case. It aims at an audit tool that needs no inside access. Such a tool helps check privacy and copyright claims.
Catalog record
| CREATOR | Jiatong Yi, Yanyang LiThe Chinese University of Hong Kong {jtyi2, yyli21}@cse.cuhk.edu.hk |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-11314
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Membership Inference on LLMs in the Wild Jiatong Yi, Yanyang LiThe Chinese University of Hong Kong {jtyi2, yyli21}@cse.cuhk.edu.hk Abstract Membership Inference Attacks (MIAs) act as a crucial auditing tool for the opaque training data of Large Language Models (LLMs). However, existing techniques predominantly rely on inaccessible model internals (e.g., logits) or suffer from poor generalization across domains in strict black-box settings where only generated text is available. In this work, we propose SimMIA, a robust MIA framework tailored for this text-only regime by leveraging an advanced sampling strategy and scoring mechanism. Furthermore, we present WikiMIA-25, a new benchmark curated to evaluate MIA performance on modern proprietary LLMs. Experiments demonstrate that SimMIA achieves state-of-the-art results in the black-box setting, rivaling baselines that exploit internal model information 1 1 1 https://github.com/simmia2026/SimMIA . (a) SaMIA (b) SimMIA Figure 1: Comparison between SaMIA ( 18 ) , a representative black-box MIA baseline, and our proposed SimMIA. Given a text x : (a) SaMIA samples multiple candidate continuations conditioned on the underlined prefix, computes ROUGE-N scores ( 22 ) between each candidate and x , and aggregates these scores to determine membership; (b) SimMIA instead performs word-by-word sampling for every prefix, computes word-level scores based on sampled and the corresponding words from x using word embeddings, and aggregates these scores to derive the final membership prediction. denotes the empirical frequency of the corresponding word, while is the empirical frequency if additionally conditioning on non-member data. 1 Introduction The unprecedented success of Large Language Models (LLMs) is fueled by pretraining on massive-scale corpora ( 34 ; 25 ; 28 ) . However, as state-of-the-art (SOTA) models become increasingly opaque, details regarding their training data are rarely disclosed. This lack of transparency poses significant challenges for auditing potential copyright infringement, privacy leakage, and data contamination. A promising avenue to address these concerns is pretraining data detection ( 26 ; 11 ) , which typically employs Membership Inference Attacks (MIAs) to determine whether a specific text was part of a model’s training set ( 27 ) . The majority of existing MIA techniques for LLMs are logit-based , relying on the assumption that the model exposes per-token probabilities ( 36 ; 33 ) . While effective for open-weight models, this assumption breaks down for API-accessed LLMs, where typically only the generated text is returned. To mitigate reliance on model internals, early explorations proposed to estimate logits via linear regression ( 15 ) or infer membership based on tokenizer distinctiveness ( 29 ) . Despite their utility, these methods often require knowledge of the target model’s tokenizer, information that remains proprietary for many closed-source systems. Therefore, recent research has pivoted toward a strictly black-box setting, where the attacker has access only to generated text ( 8 ; 18 ) . Current black-box methods predominantly rely on approximating likelihoods via N-gram matching. However, we identify that these approximation methods suffer from two critical limitations: distribution drift , where long-form generation diverges from the original distribution, and signal sparsity , where exact N-gram matching fails to capture semantic retention, offering unstable estimates across domains. In this work, we propose SimMIA, a novel framework that significantly advances MIA performance in the black-box setting. Unlike prior works that sample full sequences, SimMIA employs a word-by-word sampling strategy to eliminate error propagation (mitigating drift) and introduces a semantic scoring mechanism that captures soft membership signals beyond exact token matching (mitigating sparsity). We evaluate SimMIA on two standard benchmarks, WikiMIA ( 26 ) and MIMIR ( 9 ) , across four representative open-source families. Our results demonstrate that SimMIA outperforms existing black-box baselines by a significant margin of 15.7 AUC points on average and achieves parity with, or even surpasses, many logit-based approaches. To further validate SimMIA’s effectiveness on modern proprietary LLMs, we follow WikiMIA-24 ( 12 ) to curate WikiMIA-25, a dataset derived from 2025 Wikipedia dumps. Experiments on WikiMIA-25 with the four latest LLMs show that SimMIA establishes a new SOTA in the black-box setting, offering a practical tool for auditing the black-box LLMs of today. 2 SimMIA 2.1 Problem Statement Let M be a target LLM trained on a dataset D . Given a text x , the goal of MIA is to determine whether x\in D . Existing LLM MIAs formalize this as constructing a scoring function over x , where a higher score indicates a higher likelihood of membership. Based on the information exposed by M , we distinguish between two threat models: • Gray-box Setting : In this scenario, the attacker has access to the model’s internal tokenization results and even per-token logits. Most existing MIAs operate under this assumption ( 26 ; 23 ; 33 ; 15 ) . • Black-box Setting : This setting represents the restricted access typical of commercial APIs. The attacker has no access to model weights, logits, or the tokenization results. The attacker can only observe the generated textual continuation ( 8 ; 18 ) . In this work, we focus on the strictly constrained black-box setting, aiming to perform MIA solely through text-based interaction. 2.2 Method The core idea behind SimMIA is that if a text x has appeared in the training data of M , then continuations sampled from M given any prefix of x should be more semantically consistent with the ground truth suffix than those for unseen examples. SimMIA operationalizes this idea in three stages: (1) word-by-word sampling, (2) computation of word-level scores, and (3) relative aggregation of these scores into a final membership score. Figure 1 illustrates the workflow of SimMIA and compares it with the SOTA black-box MIA baseline. Word-by-Word Sampling Previous sampling-based MIA methods ( 8 ; 18 ) typically generate multiple full continuations x_{\geq i} conditioned on a fixed prefix x_{<i} and use them to estimate likelihood-based scores. However, such approaches suffer from distribution shift : as the generation proceeds, later tokens depend on both the true prefix and the model’s self-generated outputs, making the estimated distribution deviate from that of the original text x . SimMIA mitigates this issue by sampling only the immediate next word, which avoids self-conditioning and provides an unbiased estimate of the model’s local behavior without increasing the sampling cost. Specifically, we first tokenize x into a sequence of words x_{1},x_{2},\ldots,x_{L} using a standard word tokenizer ( 3 ) , where L is the number of words. Starting from the second word, SimMIA draws N samples of the next word \hat{x}_{i} from the conditional distribution M(\cdot\mid x_{<i}) , and computes a word-level score for x_{i} based on these samples. Semantic Scoring A straightforward method for scoring the target word x_{i} involves estimating its empirical probability using a set of generated samples \{\hat{x}^{(j)}_{i}\}_{j=1}^{N} . This formulation aligns with conventional logit-based baselines ( 35 ; 23 ; 26 ; 33 ) : s^{*}(x_{i}\mid x_{<i})=\frac{1}{N}\sum_{j=1}^{N}\mathbb{I}\left[x_{i}=\hat{x}^{(j)}_{i}\right], (1) where \mathbb{I}[\cdot] is the indicator function. Laplace smoothing is applied to prevent zero-probability estimates. We denote this version as SimMIA ∗ . However, this approach relies on a hard indicator, which is inherently sparse due to the high dimensionality of the vocabulary. This way also fails to utilize the information present in sampled words that do not exactly match x_{i} . Consequently, it suffers from high variance