Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistants
Yash Vekaria, Aurelio Loris Canino, Jonathan Levitsky, Alex Ciechonski, Patricia Callejo, Anna Maria Mandalari, Zubair Shafiq · 2025
Read Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistants by Yash Vekaria, Aurelio Loris Canino, Jonathan Levitsky, Alex Ciechonski, Patricia Callejo, Anna Maria Mandalari, Zubair Shafiq (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16586.txt.
Introduction
Yash Vekaria and six co-authors present a 2025 audit of privacy in generative AI browser assistants. These add-ons answer questions, sum up pages, and can fill forms on their own. The team studied the ten most popular ones, tracking their network traffic and probing them with a new test frame.
Most assistants send work to server APIs that can wake on their own, with no user action. When active, they ship page content, often the full page code and at times form entries, to their own servers. Some share user IDs and prompts with third-party trackers. Collection went on even on pages with health data or ID numbers. Several tools inferred age, gender, income, and interests, then shaped replies to that profile.
Catalog record
| CREATOR | Yash Vekaria, Aurelio Loris Canino, Jonathan Levitsky, Alex Ciechonski, Patricia Callejo, Anna Maria Mandalari, Zubair Shafiq |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-16586
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistants Yash Vekaria, Aurelio Loris Canino, Jonathan Levitsky, Alex Ciechonski, Patricia Callejo, Anna Maria Mandalari, Zubair Shafiq Abstract Browser assistants have started to integrate powerful capabilities of GenAI in web browsers to offer functionalities such as question answering, content summarization, and agentic web navigation. These assistants, available today as browser extensions, raise significant privacy concerns because they can track detailed browsing activity (e.g., searches, clicks) and autonomously perform tasks such as form filling. In this paper, we analyze the design and behavior of GenAI browser extensions, focusing on how they collect, process, and share user data, and whether they profile users based on explicit or inferred demographic attributes and interests. We develop a novel prompting framework and perform network traffic analysis to audit the nine GenAI browser assistants for tracking, profiling, and personalization. We find that GenAI browser assistants typically rely on server-side APIs rather than local models, and can be invoked automatically without explicit user interaction. GenAI browser assistants often collect and share full webpage content, including the HTML DOM and user form inputs in some cases, with their first-party servers. Some also share identifiers and user prompts with third-party trackers such as Google Analytics. This data collection and sharing happens even on pages containing sensitive information, such as health records or personal information such as names or social security numbers entered in a web form. Moreover, several GenAI browser assistants infer attributes (e.g., age, gender, income, interests) and use them to personalize responses across browsing contexts. Our findings show that GenAI browser assistants collect and share personal and sensitive information for profiling and personalization, highlighting the need for safeguards as they increasingly mediate web browsing. 1 Introduction With recent advances in natural language processing, Large Language Models (LLMs) trained on massive datasets and built with billions of parameters have significantly improved their ability to understand context and generate content across modalities, including text, images, and videos [ 26 ] . As the foundational technology behind many Generative Artificial Intelligence (GenAI) systems, LLMs now power a wide range of applications such as chatbots and workflow automations. Popular search engines have also begun integrating LLMs. For instance, Google Search now displays Gemini-generated overviews in response to user queries [ 39 ] , while Microsoft Bing’s Copilot uses models from OpenAI [ 25 ] . However, these search engines are limited to analyzing activity on their own platforms, restricting their visibility of a user’s browsing activity across the web. To address this limitation, GenAI browser assistants have emerged to use LLMs to enhance a user’s overall browsing experience across the web. GenAI browser assistants are implemented as browser extensions, giving them access to nearly everything a user does in their browser. By operating as a browser extension, these assistants can track a user’s browsing activity across websites such as web pages visited, buttons clicked, search terms and other personal information typed into form fields. This browsing activity provides context that assistants can use to build detailed user profiles. While this enables personalized responses by these assistants, it also raises serious privacy concerns. For example, browsing data and user profiles can be collected, shared, and repurposed for other usecases such as online behavioral advertising. Despite high operational costs [ 19 ] , most LLM platforms do not yet rely on ads for monetization. However, this shift is imminent [ 42 , 47 , 43 ] . It is important to understand the privacy risks posed by tracking, profiling, and personalization by GenAI-based browser assistants. Millions of users already use these assistants, yet little is known about how they are designed, what information they collect and share, and how they process it. To this end, we propose a novel framework to systematically audit GenAI browser assistants, guided by the following research questions: RQ1. How is the architecture of GenAI browser assistants designed? We analyze network traffic of these browser assistants to examine key design choices: (1) backend model capabilities, (2) response architecture, (3) context isolation across interactions, and (4) response variability. RQ2. Do GenAI browser assistants collect and share user information? We analyze network traffic of these browser assistants to examine how user data is handled, focusing on: (1) implicit and explicit collection and sharing, (2) of data with first-party and third-party servers, and (3) differences across public and private online spaces. RQ3. Do GenAI browser assistants profile users based on their browsing behavior to personalize responses ? We develop a prompting framework to evaluate profiling and personalization, focusing on: (1) whether assistants infer user attributes such as location, age, gender, income, and interests from browsing activity, and (2) whether they use this information to personalize responses within and across browsing contexts. We summarize our key contributions and findings below: \bullet Architecture. We analyze nine popular GenAI browser assistants and find that all but one rely on server-side response generation; only a single assistant performs response generation on the client side. Some assistants are automatically invoked when a user enters a query on a search engine. Furthermore, seven assistants isolate context across browsing sessions and tabs, while the remaining two do not do so. \bullet Tracking. Using assistants in private online spaces resulted in data collection ranging from partial webpage content to full DOM snapshots. Merlin was observed to extract form inputs, leading to the collection and sharing of sensitive information, including medical records from a university health portal ( hem.ucdavis.edu ), academic records from Canvas (from canvas.edu ), and a Social Security number entered on an IRS website irs.gov ). Sider and Merlin shared chat and user identifiers with google-analytics.com while TinaMind shared them with analytics.google.com . Merlin also shared raw user queries with Google Analytics, which could potentially be used for tracking and personalization across Google platforms. Chat histories were shared with first-party servers by four assistants. Moreover, Harpa and Copilot stored the complete chat history in the extension’s background service worker using IndexedDB, indicating that these histories persist across browsing sessions. \bullet Profiling and Personalization. Two extensions (Monica, and Sider) demonstrated profiling based on all five user attributes (location, age, gender, income, and interests) and responded to both in-context and out-of-context personalization prompts. In contrast, Perplexity and TinaMind showed no strong evidence of profiling or personalization, while Harpa exhibited only in-context personalization behavior. 2 Background We provide background of GenAI in Section 2.1 and its application in browser assistants in Section 2.2 . 2.1 Generative AI How do GenAI systems work? Generative AI (GenAI) systems typically rely on large language models (LLMs), which operate by iteratively predicting the next token in a sequence based on the context of preceding tokens [ 52 ] . These models are built on transformer architectures, where a self-attention mechanism enables the model to assign varying importance to different parts of the input sequence [ 44 ] . When a user inputs a query, the model encodes the input into an embedding space and generates a probability distribution over possible next tokens. It then selects a token based on this distribution and repeats the process until the output is complete. This approach enables GenAI systems to provide coherent and context-aware responses to