Investigating In-Context Privacy Learning by Integrating User-Facing Privacy Tools into Conversational Agents

Mohammad Hadi Nezhad, Francisco Enrique Vicente Castro, Ivon Arroyo · 2026

Read Investigating In-Context Privacy Learning by Integrating User-Facing Privacy Tools into Conversational Agents by Mohammad Hadi Nezhad, Francisco Enrique Vicente Castro, Ivon Arroyo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19416.txt.

Introduction

Nezhad and co-authors test privacy tools that teach users while they chat with an agent. Many users hold stale or wrong views on what chatbots do with private data. A side panel in a mock chat app flags posts that hold sensitive facts. It warns the user, offers safe actions, and links to privacy FAQs. Groups with and without the panel did two task rounds each. Survey and think-aloud data show how views shifted with use. Design traits that helped or blocked self-protection are drawn out.

Catalog record

CREATORMohammad Hadi Nezhad, Francisco Enrique Vicente Castro, Ivon Arroyo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66724
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19416

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Investigating In-Context Privacy Learning by Integrating User-Facing Privacy Tools into Conversational Agents Mohammad Hadi Nezhad, Francisco Enrique Vicente Castro, Ivon Arroyo Abstract Supporting users in protecting sensitive information when using conversational agents (CAs) is crucial, as users may undervalue privacy protection due to outdated, partial, or inaccurate knowledge about privacy in CAs. Although privacy knowledge can be developed through standalone resources, it may not readily translate into practice and may remain detached from real-time contexts of use. In this study, we investigate in-context, experiential learning by examining how interactions with privacy tools during chatbot use enhance users’ privacy learning. We also explore interface design features that facilitate engagement with these tools and learning about privacy by simulating ChatGPT’s interface which we integrated with a just-in-time privacy notice panel. The panel intercepts messages containing sensitive information, warns users about potential sensitivity, offers protective actions, and provides FAQs about privacy in CAs. Participants used versions of the chatbot with and without the privacy panel across two task sessions designed to approximate realistic chatbot use. We qualitatively analyzed participants’ pre- and post-test survey responses and think-aloud transcripts and describe findings related to (a) participants’ perceptions of privacy before and after the task sessions and (b) interface design features that supported or hindered user-led protection of sensitive information. Finally, we discuss future directions for designing user-facing privacy tools in CAs that promote privacy learning and user engagement in protecting privacy in CAs. 1 Introduction Protecting privacy during conversational agent (hereafter CAs or chatbots) use requires users to continuously assess what information—about themselves or others— “is appropriate, or fitting, to reveal in a particular context” [ 35 ] . However, users often disclose sensitive information that may not be necessary for completing tasks [ 43 , 28 , 30 ] . Even when such information is relevant, disclosure decisions are not always made by a careful weighing of immediate benefits against potential future privacy risks. Examples include treating CAs as therapists and sharing emotional or behavioral details, providing medical or fitness information to receive advice, or uploading lengthy documents (e.g., emails or contracts) for summarization or search [ 43 , 30 ] . In pursuing these goals, users may overlook the sensitivity of the information they are disclosing, be unaware of available actions to protect their information, or find such protective actions time-consuming or inconvenient. As a result, they may undervalue privacy protection and its importance during CA interactions. This makes it essential to support users in more context-aware and privacy-conscious decision-making. Given these concerns, it is essential to support different user groups in developing a stronger understanding of privacy in the context of CAs, as supporting them can promote greater and more meaningful engagement with privacy-conscious behaviors [ 44 , 9 ] . While such understanding can be fostered through standalone learning resources (e.g., educational tools or academic programs) [ 13 , 14 ] , it may not easily translate into practice, as it is often detached from the real-time contexts of use in which privacy decisions must be made (e.g., what information to protect and how, depending on the tasks and constraints including time and effort) and where timely awareness and support are most critical. We thus explore how exposing users to privacy tools during realistic chatbot use scenarios can support experiential, in-context learning about privacy, while also exploring user interface and experience (UI/UX) design features that encourage user engagement with privacy protections and facilitate learning. We first examine key aspects of privacy perceptions among undergraduate and master’s students in a Computer Science (CS) program in the US (participant details in Section 3.3 ). We then analyze how these perceptions evolve through interactions with a just-in-time privacy notice panel (Section 3.1 ) that embeds learning opportunities directly within CA interfaces. To this end, we integrated a simulated ChatGPT interface with our privacy panel that intercepts interactions immediately after users attempt to submit a message containing sensitive information. Our panel (1) detects and informs users about potentially sensitive information in their messages (e.g., names, physical addresses); (2) offers features for applying anonymization strategies (retracting, faking, generalizing); (3) surfaces two built-in ChatGPT privacy controls (disabling memory, opting out of data sharing); and (4) includes FAQs that describe how to think about sensitive information and factors to consider when making disclosure decisions. We analyzed our panel’s effectiveness through a five-phase study (Section 3.2 ) with ten participants: (1) a pre-test survey examining baseline privacy perceptions; (2–3) two study sessions to complete tasks using the simulated chatbot with and without our privacy panel; (4) an immediate post-test survey collecting reflections on the task sessions; and (5) a delayed post-test survey to examine changes in privacy perceptions. Our task and study design closely approximate realistic chatbot use for our participant group by incorporating a range of real-world use scenarios involving sensitive information (e.g., writing emails, searching documents), withholding the study’s privacy goals until after participation to reduce bias, and allowing participants to freely direct their own interactions (e.g., prompting, disclosure decisions). Thus, we examine the following research question (RQ): RQ1. Before and after completing tasks using the chatbot, what did students think about— 1.1. what they consider as sensitive information? 1.2. the importance of protecting sensitive information when using chatbots? 1.3. what they can do to protect sensitive information when using ChatGPT? Given the critical role of UI/UX design in users’ engagement with and adoption of interactive tools, there are gaps and opportunities to better understand how interface design decisions support or hinder the protection of sensitive information when using CAs. Thus, we adopt an exploratory approach to examine such design features in our privacy panel by analyzing students’ think-aloud transcripts and interface interactions from task sessions and responses to open-ended survey questions. Therefore, we examine the following RQ: RQ2. What interface design features of the privacy panel support or hinder the protection of sensitive information? Through a detailed qualitative analysis of data (Section 3.4 ), we present findings on how our participants conceptualized sensitive information (Section 4.1 ), viewed the importance of protecting sensitive information during CA use (Section 4.2 ), and what protective actions they thought they can take (Section 4.3 ). We also report findings on interface design decisions that supported or hindered users’ protection of sensitive information when interacting with our privacy panel (Section 4.5 ). We then discuss opportunities for future work on enhancing in-context support and experiential privacy learning (Section 5.1 ), designing UI/UX to promote engagement and learning in user-facing privacy tools (Section 5.2 ), supporting users in navigating privacy trade-offs (Section 5.3 ), and identifying learning opportunities for students (Section 5.4 ). 2 Background and Related Work 2.1 User Knowledge and Agency Over Privacy When Using Conversational Agents (CA) Users need to understand and consider various aspects of privacy in their decision-making to meaningfully engage in privacy-protective behaviors in CAs use. Decisions about what to disclose or withhold may be shaped by several factors, including individual preferences (e.g., sensitivity of financial information may differ across individuals [ 5 ] ), the features of the technology (e.g., privacy policies, security safeguards), and the perceived trustworthiness of the technology providers in protecting user privacy [ 25 , 37 , 43 , 31

Read the complete text