From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents
Myeongseob Ko, Jihyun Jeong, Sumiran Singh Thakur, Gyuhak Kim, Ruoxi Jia, [0.7em] Virginia Tech, [0.3em] Equal contribution · 2026
Read From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents by Myeongseob Ko, Jihyun Jeong, Sumiran Singh Thakur, Gyuhak Kim, Ruoxi Jia, [0.7em] Virginia Tech, [0.3em] Equal contribution (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18382.txt.
Introduction
Ko and co-authors evaluate inference-driven de-anonymization by LLM agents from weak cues. Removing explicit identifiers was long thought to protect privacy in shared data. Agents now join scattered, non-identifying cues with public evidence to rebuild real identities. In the sparsest Netflix Prize setting, agents rebuilt 79.2% of identities against a 56.0% classical baseline. They also link people without an explicit request, and more often when asked.
Catalog record
| CREATOR | Myeongseob Ko, Jihyun Jeong, Sumiran Singh Thakur, Gyuhak Kim, Ruoxi Jia, [0.7em] Virginia Tech, [0.3em] Equal contribution |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72010 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.18382
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents Myeongseob Ko, Jihyun Jeong, Sumiran Singh Thakur, Gyuhak Kim, Ruoxi Jia, [0.7em] Virginia Tech, [0.3em] Equal contribution Abstract Anonymization is widely treated as a practical safeguard: once explicit identifiers are removed, linking records back to specific individuals is assumed to become substantially more difficult. Historically, this difficulty did not arise because re-identification was impossible, but because it was costly, often requiring domain expertise, tailored algorithms, and labor-intensive manual corroboration. We study a growing privacy risk: LLM-based agents may weaken this practical barrier. If an agent can combine scattered, individually non-identifying cues with publicly available information to reconstruct specific identities, then anonymized records may become linkable without bespoke engineering. Such linkage can emerge not only under explicit re-identification requests, but also while the agent is performing otherwise benign tasks. This challenges a long-standing assumption underlying anonymization: that meaningful identity linkage requires specialized effort. To study this phenomenon systematically, we evaluate identity reconstruction across three settings. First, we revisit classical linkage scenarios (Netflix and AOL) to test whether modern agents can reproduce core linkage behaviors without bespoke engineering. Second, we introduce InferLink , a controlled benchmark that varies task framing, shared cues, and attacker knowledge. Third, we examine modern text-rich artifacts to assess whether similar risks arise in realistic contemporary settings. In the Netflix Prize deanonymization setting, agents reconstruct 792 out of 1,000 identities (79.2%), compared to 56.0% for a classical matching baseline. In InferLink , agents identify the correct individual even when no re-identification is explicitly requested, with success rates rising sharply under explicit prompts. In contemporary text-rich traces, agents resolve anonymized profiles to real-world individuals by corroborating contextual cues with public evidence. Taken together, these results suggest that identity inference—not only information disclosure—should be treated as a first-class privacy risk in agentic systems. Privacy evaluations should therefore measure not only what information is accessed or revealed, but also what identities can be inferred. Code github.com/jihyun-jeong-854/InferLink Project Page jihyun-jeong-854.github.io/InferLink Figure 1: Overview of Inference-Driven Linkage. We show that an LLM agent can reconstruct a specific identity ( \hat{\imath} ) from fragmented information. (Left) Anonymized Artifacts ( D_{\text{anon}} ): Sources such as ChatGPT logs, AOL search histories, and interview transcripts contain fragmented, individually non-identifying cues that can jointly form an identity-relevant profile within a given artifact. (Right) Auxiliary Context ( D_{\text{aux}} ): Auxiliary information may be either provided directly or retrieved from public sources (e.g., the web, social media, and news), and serves as corroborating evidence. (Center) Agentic Inference: The agent combines weak cues from anonymized artifacts with corroborating auxiliary context to form a coherent identity hypothesis. 1 Introduction Anonymization is widely treated as a practical safeguard: once names and other explicit identifiers are removed, records are presumed to be difficult to trace back to specific individuals. Historically, this assumption held not because re-identification was impossible, but because it was costly. Successful linkage required domain expertise, tailored algorithms, and labor-intensive heuristics. In the Netflix Prize case, attackers engineered specialized similarity metrics to match sparse rating traces across datasets ( 14 ) . In the AOL search log incident, investigators manually narrowed candidates and corroborated clues from public sources ( 2 ) . These technical and human costs acted as a practical barrier. This historical reliance on the high cost of linkage motivates the core question we study: whether LLM-based agents possess the capabilities required to weaken this practical barrier. If an agent can combine heterogeneous signals, generate candidate hypotheses, and seek corroborating evidence, then anonymized records may become linkable without bespoke engineering. This possibility is concerning because it can arise even when re-identification is not the user’s objective, as candidate generation and corroboration may occur as a byproduct of completing otherwise benign tasks. We term this failure mode inference-driven linkage : a privacy failure in which an agent reconstructs a specific real-world identity by combining non-identifying cues from anonymized artifacts with corroborating signals from auxiliary context. Despite this capability shift, existing agent privacy evaluations do not directly capture this risk. Frameworks such as PrivacyLens and AgentDAM evaluate privacy in action by tracking whether agents access, use, or explicitly disclose sensitive information while completing tasks ( 18 ; 21 ) . These evaluations are essential for measuring data minimization and overt leakage, but they are not designed to measure inference-driven linkage : whether an agent can transform anonymized traces into a concrete identity hypothesis. Recent concurrent work shows that LLMs and web-capable agents can indeed re-identify individuals in real-world settings, including linking anonymized interview transcripts to named researchers ( 12 ) and deanonymizing pseudonymous users across online platforms ( 11 ) . However, these studies do not systematically examine how identity reconstruction changes as the overlap between the anonymized and auxiliary sources changes, or as task framing and attacker knowledge vary. To address the gap, we introduce a systematic evaluation framework for inference-driven identity reconstruction. Given anonymized artifacts and auxiliary context, we test whether an agent produces a specific identity hypothesis under varying conditions. We apply this framework in three complementary settings. First, we revisit classical linkage scenarios such as Netflix and AOL to test whether modern agents reproduce core linkage behaviors without bespoke engineering. Second, we construct InferLink , a controlled benchmark that varies shared-cue type, task framing (benign analysis versus explicit re-identification), and attacker knowledge, allowing us to isolate the factors that drive linkage success. Third, we examine modern text-rich artifacts to assess whether the same linkage behavior persists in realistic usage traces. Our findings show that modern agents can reconstruct identity across classical, controlled, and modern trace settings without bespoke linkage engineering. In the Netflix linkage setting, agents reconstruct 792 out of 1,000 identities (79.2%, averaged over three resamples), compared to 560 out of 1,000 (56.0%) for the classical matching baseline ( 14 ) . In InferLink , agents sometimes identify the correct individual even when no re-identification is requested (e.g., 16/20 cases for Claude 4.5). Under explicit prompts, successful linkage becomes highly prevalent (e.g., 19/20 for GPT-5). In modern text-rich artifacts, agents resolve anonymized profiles to specific individuals by corroborating contextual cues with public information. We also show that privacy-aware system prompts can suppress linkage in the controlled benchmark, but at a measurable cost to task performance, revealing a concrete privacy–utility trade-off. Our contributions are: • We formalize inference-driven linkage as a distinct privacy failure mode for LLM-based agents, in which identity is reconstructed from fragmented, individually non-identifying cues rather than through direct private information disclosure. • We design InferLink , a controlled benchmark that isolates how task framing, available shared cues, and attacker knowledge affect identity reconstruction, enabling systematic analysis of when linkage emerges and how it changes across conditions. • We provide a unified evaluation across classical incidents, controlled experiments, and modern text-rich artifacts, showing how the same underlying risk appears in historically documented cases, controlled settings, and realistic contemporary traces. 2 Background and Related Work 2.1 Privacy Risks in LLMs Prior work on LLM privacy has focused primarily on training-time exposure. In this setting, models may memorize sensitive information and enable downstream recovery by attackers. Representative threats include membership inference ( 10 ; 4 ; 19 ) and training-data extraction ( 5 ; 15