Network-Level Prompt and Trait Leakage in Local Research Agents

Hyejun Jeong, Mohammadreza Teymoorianfard, Abhinav Kumar, Eugene Bagdasarian · 2025

Read Network-Level Prompt and Trait Leakage in Local Research Agents by Hyejun Jeong, Mohammadreza Teymoorianfard, Abhinav Kumar, Eugene Bagdasarian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-20282.txt.

Introduction

Hyejun Jeong and a team of co-authors wrote a security paper in two thousand twenty-five about research agents that search the web. People run these tools on their own machines to keep their questions private. Yet the tools visit many sites per request, in a clear timing pattern.

The authors show that a watcher who sees only site addresses and timing can guess much of a user's prompt. In tests, the attack kept most of the prompt's facts and topic. Over many sessions, it could also guess traits of the user. Ways to hide the pattern cut the attack, with little loss in use.

Catalog record

CREATORHyejun Jeong, Mohammadreza Teymoorianfard, Abhinav Kumar, Eugene Bagdasarian
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE103 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-20282

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Network-Level Prompt and Trait Leakage in Local Research Agents Hyejun Jeong, Mohammadreza Teymoorianfard, Abhinav Kumar, Eugene Bagdasarian Abstract We show that Web and Research Agents (WRAs)—language model-based systems that investigate complex topics on the Internet—are vulnerable to inference attacks by passive network adversaries such as ISPs. These agents could be deployed locally by organizations and individuals for privacy, legal, or financial purposes. Unlike sporadic web browsing by humans, WRAs visit 70 − 140 70{-}140 domains with distinguishable timing correlations, enabling unique fingerprinting attacks. Specifically, we demonstrate a novel prompt and user trait leakage attack against WRAs that only leverages their network-level metadata (i.e., visited IP addresses and their timings). We start by building a new dataset of WRA traces based on user search queries and queries generated by synthetic personas. We define a behavioral metric (called OBELS) to comprehensively assess similarity between original and inferred prompts, showing that our attack recovers over 73% of the functional and domain knowledge of user prompts. Extending to a multi-session setting, we recover up to 19 of 32 latent traits with high accuracy. Our attack remains effective under partial observability and noisy conditions. Finally, we discuss mitigation strategies that constrain domain diversity or obfuscate traces, showing negligible utility impact while reducing attack effectiveness by an average of 29%. † † Code available at https://github.com/umass-aisec/wra Figure 1: Overview of the attack. 1 Introduction Web and Research Agents (WRAs) are reshaping how users interact with information. Unlike traditional assistants that answer isolated queries, these Large Language Model (LLM)-powered systems autonomously plan, browse, and synthesize knowledge across the web [ 30 ] . Major AI providers are integrating such agents directly into their flagship assistants or browsers, including OpenAI [ 55 ] , Google Gemini [ 23 ] , Grok [ 85 ] , Mistral [ 41 ] , and Perplexity [ 58 ] . Open-source counterparts such as LangChain’s Local Deep Researcher [ 35 ] , GPT Researcher [ 21 ] , and Agent Laboratory [ 65 ] are also widely adopted in academic and developer communities. Their growing deployment in assistants, productivity platforms, and enterprise environments, particularly within privacy-sensitive sectors such as healthcare, law, and finance, raises urgent concerns about their security and privacy implications. While on-device execution through locally-deployable models and TLS-encrypted access could suggest privacy guarantees, this perception is misleading. Yet WRAs leave unavoidable, structured metadata traces that can reveal sensitive user intent, creating an underexplored leakage vector. As autonomous agent systems, WRAs inherit vulnerabilities from both LLMs and traditional web browsing, such as system prompt extraction [ 31 ] , model memorization [ 12 ] , metadata exposure [ 69 ] , and behavioral fingerprinting [ 8 ] . Yet they also expose a distinct and underexplored vulnerability: their behavioral traces. They reflect the agent’s internal reasoning process and can reveal sensitive or stigmatized user intent [ 52 , 73 ] . Even if prompts and page contents remain fully encrypted, externally visible metadata, such as domain names, access order, payload size, and timing, can silently expose user intent [ 52 , 73 ] . Because WRAs autonomously execute multi-step research plans [ 90 ] , they produce cascades of semantically related domain visits in dense temporal bursts [ 58 ] . These structured, high-throughput access patterns [ 17 , 42 ] are unlike human browsing behavior and make WRAs particularly amenable to profiling and inference attacks [ 17 , 42 ] while introducing new risks through autonomous, high-throughput interaction with live websites. We adopt a realistic threat model in which a passive adversary, such as an ISP, enterprise firewall, or local network operator, can observe only domain-level traffic metadata, without access to prompts, page contents, or model internals. Despite this extremely limited visibility, we show that adversaries can reliably recover what users asked and who they are. Motivating Examples. Consider a user seeking reproductive health information: their research agent may visit clinics, support forums, and medical sites. Even without access to page contents, these traces alone can expose the agent’s internal reasoning process and deeply stigmatized intent. As WRAs become integrated into daily decision-making, their behavioral traces present a new leakage channel. We show that WRA visits 70-140 distinct domains for each query in a short period of time, making it an easy target to isolate. In essence, privacy failures emerge not from what agents say, but from how they act. Our Work. We present privacy leakage attacks that exploit metadata traces of web and research agents deployed locally. The attacks include (1) Prompt Recovery , which recovers a user’s original prompt from observed domain sequences, and (2) Trait Inference , which profiles latent attributes (e.g., gender, ideology) from the agent’s multi-session browsing patterns. The adversary trains an inference model using task-specific strategies such as In-Context Learning (ICL) and fine-tuning to map browsing traces to inference targets. Our results show that the attacks remain effective even when 40% of the traces are masked or fully obfuscated, revealing that encrypted content alone cannot guarantee privacy. We observe that WRAs visit dozens of marginal or redundant domains that do not improve the final report but still expand the adversary’s visibility. To mitigate these risks, we propose and evaluate practical defenses that either hide traces , by injecting LLM-generated decoy prompts and virtual personas to confuse inference, or block traces , by redirecting tasks to multipurpose sources or LLM knowledge, avoiding uniquely identifying domains. Our contributions are as follows: • New attack surface: We introduce metadata-based privacy attacks from behavioral traces produced by WRAs and propose a general two-stage pipeline for prompt recovery and trait inference. • Benchmark dataset and OBELS metric: We construct and will release a dataset linking domain traces to prompts and persona traits, and introduce OBELS, a new ontology-aware metric for evaluating privacy leakage through behavioral traces. • Ablation and countermeasures: We conduct extensive experiments across agents, LLMs, and masking conditions, and evaluate mitigation strategies that constrain domain diversity or obfuscate traces while negligibly affecting utility. 2 Background and Related Work 2.1 AI, Web, and Research Agents Standalone LLMs to AI Agents. LLMs excel at language understanding but are limited as static knowledge systems [ 2 , 76 ] . They cannot access real-time information, call external APIs, or execute multi-step plans, but produce responses based only on their pretrained knowledge. Early solutions such as RAG [ 38 , 24 ] and tool-augmented LLMs [ 63 , 68 , 60 , 16 ] improved grounding and tool use, but remain constrained to pre-indexed corpora and invoking tools only when explicitly instructed. AI agents address this by embedding LLMs into a perception–action loop [ 86 , 70 , 79 ] , where the model plans, executes, and adapts across multiple steps, enabling autonomous reasoning and dynamic interaction with external environments. Web Agents. Generic agents can reason about which queries to run, but often lack robust capabilities for controlling browsers, parsing dynamic content, handling JavaScript-heavy pages, or managing session state across multiple sites. Web agents integrate LLM planning with browser control: issuing search queries, navigating between sources, extracting content, and deciding which actions to perform based on the evolving context [ 17 ] . Frameworks like AutoGen [ 84 ] and Browser-Use

Read the complete text