Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI

Isadora Krsek, Anubha Kabra, Yao Dou, Tarek Naous, Laura A. Dabbish, Alan Ritter, Wei Xu, Sauvik Das · 2024

Read Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI by Isadora Krsek, Anubha Kabra, Yao Dou, Tarek Naous, Laura A. Dabbish, Alan Ritter, Wei Xu, Sauvik Das (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-15047.txt.

Introduction

On sites like Reddit, people share hard parts of their lives with strangers, but a post can reveal more than they intend. Isadora Krsek and co-authors studied that risk in 2024 with 21 Reddit users. Each user ran a language model that flags risky disclosures on two of their own posts, then spoke about what the tool got right and wrong.

Users liked the tool despite its flaws, saying it caught mistakes and raised risks they had missed. Still, the team found that such tools must weigh post context, site norms, and each user's own threat model. Tools should explain a risk and inform choice, not simply tell users to stay silent.

Catalog record

CREATORIsadora Krsek, Anubha Kabra, Yao Dou, Tarek Naous, Laura A. Dabbish, Alan Ritter, Wei Xu, Sauvik Das
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-15047

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI Isadora Krsek, Anubha Kabra, Yao Dou, Tarek Naous, Laura A. Dabbish, Alan Ritter, Wei Xu, Sauvik Das Abstract . In pseudonymous online fora like Reddit, the benefits of self-disclosure are often apparent to users (e.g., I can vent about my in-laws to understanding strangers), but the privacy risks are more abstract (e.g., will my partner be able to tell that this is me?). Prior work has sought to develop natural language processing (NLP) tools that help users identify potentially risky self-disclosures in their text, but none have been designed for or evaluated with the users they hope to protect. Absent this assessment, these tools will be limited by the social-technical gap: users need assistive tools that help them make informed decisions, not paternalistic tools that tell them to avoid self-disclosure altogether. To bridge this gap, we conducted a study with N = 21 N=21 Reddit users; we had them use a state-of-the-art NLP disclosure detection model on two of their authored posts and asked them questions to understand if and how the model helped, where it fell short, and how it could be improved to help them make more informed decisions. Despite its imperfections, users responded positively to the model and highlighted its use as a tool that can help them catch mistakes, inform them of risks they were unaware of, and encourage self-reflection. However, our work also shows how, to be useful and usable, AI for supporting privacy decision-making must account for posting context, disclosure norms, and users’ lived threat models, and provide explanations that help contextualize detected risks. … DOI: XXXXXXX.XXXXXXX Conference: ACM SIGCHI Conference on Computer-Supported Cooperative Work Social Computing; October 18 — 22, 2025; Bergen, Norway ISBN: 978-1-4503-XXXX-X/18/06 CCS: Security and privacy Human and societal aspects of security and privacy Privacy protections CCS: Computing methodologies Artificial Intelligence Natural language processing 1. Introduction Every day, millions of people log onto pseudonymous online fora like Reddit to access a seemingly safe space to vent, seek support, and connect with like-minded others without the additional baggage of their offline identity. Unsurprisingly, prior work has also found that users in these pseudonymous fora are more likely to disclose highly sensitive information ( Yang et al., 2019 ) . While the benefits of these self-disclosures are apparent to users, the potential harms are more abstract and difficult to reason about. This asymmetry can lead to uninformed sharing that can result in regrets ( Wang et al., 2011 ; Sleeper, 2016 ) , self-censorship ( Das and Kramer, 2013 ; Sleeper et al., 2013 ) , and/or risks of de-anonymization ( Afualo, 2023 ; Pulliam-Moore, 2021 ; Staab et al., 2023 ) . As such, a defining dilemma of modern social networking is helping users balance the social benefits they reap through online self-disclosure with the privacy risks associated with those disclosures. Recent work ( Canfora et al., 2018 ; Zong et al., 2019 ; Morris et al., 2022 ; Guarino et al., 2022 ; Akiti et al., 2020 ) has explored the use of natural language processing (NLP) tools to help users identify potentially risky self-disclosures. However, to date, these tools have not been evaluated with the actual users that they aim to protect. Rather, most of this work has focused on improving model performance in detecting personal disclosures against benchmarks and accuracy metrics that cannot be confidently correlated with what users want ( Canfora et al., 2018 ; Morris et al., 2022 ; Guarino et al., 2022 ) . Without considering user intentions, needs, and contexts of use, NLP-powered disclosure detection tools run the risk of running headlong into Ackerman’s social-technical gap ( Ackerman, 2000 ) : i.e., the gap between what is possible technically and what is necessary socially. Users often disclose personal information online intentionally: e.g., when seeking emotional or informational support ( Record et al., 2018 ; De Choudhury and De, 2014 ; Zhang et al., 2018 ) , and to build relationships ( De Choudhury and De, 2014 ; Krasnova et al., 2010 ) . A disclosure-detection tool that fails to consider why users are disclosing personal information, and what threats they’re most concerned about in sharing that information, may fall short of helping users make informed decisions. Figure 1 . AI-powered, span-level self-disclosure detection helps users in reviewing their posts to improve privacy. Driven by this hypothesis, we ask: How can NLP-based disclosure detection tools be designed to help users make informed decisions about online self-disclosures? Taking cues from prior work in the domain of usable security & privacy, we formulate three sub-questions to better understand how users perceive the outputs of existing models, and how they might be improved: • RQ1: Given the tension between the need for self-disclosure and the risks, how can interaction with an NLP model make users aware of potential self-disclosure risks? ( Acquisti et al., 2017 ; Jeung and Huang, 2023 ) • RQ2: How should the feedback from the NLP model be framed? What level of textual granularity is helpful in communicating the abstract risks of users’ disclosures? ( Braunstein et al., 2011 ; Wang et al., 2014 ) • RQ3: Where do NLP self-disclosure detection models fall short, and how might they be further improved to better align with users’ preferences? ( Taka et al., 2023 ; Dalvi et al., 2022 ; Khritankov, 2023 ) Guided by these RQs, we conducted an in-depth interview study with 21 Reddit users. We had our participants use a state-of-the-art NLP-based disclosure detection model from prior work by Dou et al. (2023) on two self-authored posts made on and for Reddit. We chose this model because, unlike other similar models that operate at the post or sentence level ( Valizadeh et al., 2021a ; Cho et al., 2022a ) , Dou et al.’s model operates at the text-span level: i.e., it is capable of identifying specific segments of text in a broader post that constitutes a potentially risky self-disclosure (see Fig. 1 ). This higher-level of granularity let us localize user feedback to the specific words in their posts that the model estimated as constituting disclosure risk. Moreover, the model has two versions: a multi-class version (which we refer to as the categorical disclosure model ) which is capable of detecting 19 different self-disclosure categories (see Table 1 ), and a binary version that flags content as self-disclosures or not (which we refer to as the binary disclosure model ). To answer our RQs, we asked users questions about how they viewed and accepted the model’s outputs for their posts (RQ1), differences in perceptions of model utility between the categorical and binary versions of the model (RQ2), and — when users rejected model outputs — why they rejected the outputs and where they felt the model failed more broadly (RQ3). While the model was imperfect at identifying self-disclosures (with users accepting 58% of all detected disclosures overall), the majority of participants reacted positively to the model. 82% of our participants stated that they wanted to use the model outside of the study or would recommend it to others. Participants described the model’s utility as encouraging self-reflection, catching

Read the complete text